Survey of Web-based Crowdsourcing Frameworks for Subjective Quality Assessment

The popularity of the crowdsourcing for performing various tasks online increased significantly in the past few years. The low cost and flexibility of crowdsourcing, in particular, attracted researchers in the field of subjective multimedia evaluations and Quality of Experience (QoE). Since online assessment of multimedia content is challenging, several dedicated frameworks were created to aid in the designing of the tests, including the support of the testing methodologies like ACR, DCR, and PC, setting up the tasks, training sessions, screening of the subjects, and storage of the resulted data. In this paper, we focus on the web-based frameworks for multimedia quality assessments that support commonly used crowdsourcing platforms such as Amazon Mechanical Turk and Microworkers. We provide a detailed overview of the crowdsourcing frameworks and evaluate them to aid researchers in the field of QoE assessment in the selection of frameworks and crowdsourcing platforms that are adequate for their experiments

Best Practices and Recommendations for Crowdsourced QoE - Lessons learned from the Qualinet Task Force Crowdsourcing

Crowdsourcing is a popular approach that outsources tasks via the Internet to a large number of users. Commercial crowdsourcing platforms provide a global pool of users employed for performing short and simple online tasks. For quality assessment of multimedia services and applications, crowdsourcing enables new possibilities by moving the subjective test into the crowd resulting in larger diversity of the test subjects, faster turnover of test campaigns, and reduced costs due to low reimbursement costs of the participants. Further, crowdsourcing allows easily addressing additional features like real-life environments. This white paper summarizes the recommendations and best practices for crowdsourced quality assessment of multimedia applications from the Qualinet Task Force on “Crowdsourcing”. The European Network on Quality of Experience in Multimedia Systems and Services Qualinet (COST Action IC 1003, see www.qualinet.eu) established this task force in 2012 which has more than 30 members. The recommendation paper resulted from the experience in designing, implementing, and conducting crowdsourcing experiments as well as the analysis of the crowdsourced user ratings and context data

An Educated Guess on QoE in Operational Networks through Large-Scale Measurements

Network monitoring and reporting systems as well as network quality benchmarking campaigns use the Average Downlink Throughput (ADT) as the main Key Performance Indicators (KPIs) reflecting the health of the network. In this paper we address the problem of network performance monitoring and assessment in operational networks from a user-centric, Quality of Experience (QoE) perspective. While accurate QoE estimation requires measurements and KPIs collected at multiple levels of the communications stack -- including network, transport, application and end-user layers, we take a practical approach and provide an educated guess on QoE using only a standard ADT-based KPI as input. Armed with QoE models mapping downlink bandwidth to user experience, we estimate the QoE undergone by customers of both cellular and fixed-line networks, using large-scale passive traffic measurements. In particular, we study the performance of three highly popular end-customer services: YouTube, Facebook and WhatsApp. Results suggest that up to 33\% of the observed traffic flows might result in sub-optimal -- or even poor, end-customer experience in both types of network

The Impact of Adaptation Strategies on Perceived Quality of HTTP Adaptive Streaming

Changing network conditions like bandwidth fluctuations and resulting bad user experience issues (e.g. video freezes) pose severe challenges to Internet video streaming. To ad-dress this problem, an increasing number of video services utilizes HTTP adaptive streaming (HAS). HAS enables ser-vice providers to improve Quality of Experience (QoE) and resource utilization by incorporating information from dif-ferent layers. However, these adaptation possibilities of HAS also introduce new perceivable impairments such as the fluc-tuation of audiovisual quality levels over time, which in turn lead to novel QoE-related research questions. The main con-tribution of this paper is the formulation of open research questions as well as a thorough systematic user-centric anal-ysis of different quality adaptation dimensions and strate-gies. The underlying data has been acquired through two crowdsourcing and one lab study. The results provide guid-ance w.r.t. which encoding dimensions are combined best for the creation of the adaptation set and what type of adap-tation strategy should be used. Furthermore it provides in-sights on the impact of adaptation frequency and the true QoE gain of adaptation over stallings

Water Transport - The Challenge for the Automotive Industry in Slovakia

An important factor in choosing the transport mode in the automotive industry is not only the current price but also the shipping time. In some cases the latter one is even crucial and significantly affects the choice of the transport mode. Geographical location of the port of Bratislava allows its direct connection to the North and Black Sea. The study provides a comparison of transportation time from the selected Slovak car producers and shows advantages of water transport. Time of delivery is a criterion that can be easily quantified and mutually compared. The fastest transportation mode - the road transport saves the time of delivery eight times more on average than the water transport in the European transport network. The waterway, on the other hand, is cheaper and environmentally friendlier

Proposal of an Enhanced Safety System on Board of the Inland Vessel

The share of human factor in the occurrence of navigation accidents represents a high proportion of the total accident rate. Elimination of the human factor responsible for the occurrence of navigational accidents would contribute to the overall safety of navigation on inland waterways. The main causes of human-induced navigation accidents include inattention, fatigue or poor concentration on vessel control. The design of the system for increasing navigation safety consists of a system for monitoring the life functions of a crew member as well as a position of the vessel towards the fairway / shore. The first level focuses on tracking the life function of the vessel's crew / boat master. The second level addresses the position of the vessel and the possibility of increasing safety by cooperating with the data provided by Automatic Identification System - AIS. The proposal of the system represents interconnection of both levels and brings high degree of control of the vessel in extraordinary situations